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Claims 

What is claimed is: 

1 . A system that facilitates non-linear viewing of media, the system comprising: 

a scene selector that scans a digitized media and selects a scene in the digitized 

media; 

a metadata generator that produces metadata associated with the selected scene 
and relates the metadata to the selected scene; and 

an organizer that places the selected scene and the metadata in a media store to 
facilitate non-linear viewing of one or more scenes. 

2. The system of claim 1 , where the scene selector selects a scene based on at least 
one of, face recognition, item recognition, voice recognition, color recognition, mood 
recognition and theme recognition, 

3. The system of claim 2, where the scene selector selects a scene based, at least in 
part, on an input from a user. 

4. The system of claim 3, where at least one of the face recognition, item 
recognition, voice recognition, color recognition, mood recognition and theme 
recognition is adapted by a machine learning technique based, at least in part, on the 
input from the user. 

5 . The system of claim 1 , where the metadata generator produces at least one of, a 
date, a time, a length, a subject, a mood, a theme, a color, a person name, a set of person 
names, an item name and a set of item names associated with the scene. 

6. A method that facilitates non-linear viewing of media, comprising: 
selecting a scene from a set of digitized media; 

annotating the selected scene with metadata; and 

storing the annotated scene to facilitate non-linear retrieval of the annotated scene. 
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7. The method of claim 6, where selecting the scene from the set of digitized media 
comprises: 

manually scanning one or more scenes from the set of digitized media; and 
manually selecting the scene. 

8. The method of claim 6, where selecting the scene from the set of digitized media 
comprises: 

automatically scanning one or more scenes from the set of digitized media; and 
automatically selecting the scene based on at least one of face recognition, item 

recognition, voice recognition, color recognition, mood recognition and theme 

recognition. 

9. The method of claim 6, where annotating the selected scene with metadata 
comprises associating at least one of, a date, a time, a length, a subject, a mood, a theme, 
a color, a person, a set of people, an item and a set of items with the selected scene. 

10. The method of claim 6, where storing the annotated scene to facilitate non-linear 
retrieval of the annotated scene comprises storing the annotated scene in at least one of, a 
database and a datacube. 

11. A system for generating a playlist of media items to facilitate non-linear viewing 
of the media items, comprising: 

a scene retriever that retrieves one or more scenes and one or more pieces of 
annotating metadata associated with the one or more scenes from a media store; 

a metadata analyzer that analyzes the one or more pieces of annotating metadata 
to identify one or more relationships between the one or more pieces of annotating 
metadata; and 

a playlist generator that evaluates the one or more relationships and produces a 
playlist of related scenes. 
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12. The system of claim 1 1 , where the scene retriever retrieves a scene based, at least 
in part, on a value stored in the one or more pieces of annotating metadata. 

13. The system of claim 12, where the metadata analyzer computes one or more 
similarity values for the one or more relationships. 

14. The system of claim 13, where the playlist generator produces the playlist of 
related scenes based, at least in part, on the similarity values. 

15. A system for non-linear viewing of media, comprising: 

a scene retriever that retrieves one or more scenes and one or more pieces of 
annotating metadata associated with the one or more scenes from a media store; 

a metadata analyzer that analyzes the one or more pieces of annotating metadata 
to identify one or more relationships between the one or more pieces of annotating 
metadata; and 

a playlist generator that evaluates the one or more relationships and produces a 
playlist of related scenes. 

one or more viewers for viewing a scene listed in the playlist; 

one or more feedback receivers for receiving a feedback concerning the viewed 
scene; and 

a playlist updater for updating the playlist based, at least in part, on the feedback. 

1 6. The system of claim 15, where the one or more viewers comprise at least one of, 
an active device and a passive device. 

17. The system of claim 15, where the one or more viewers comprise at least one of, 
an intelligent device and a non-intelligent device. 

1 8. The system of claim 1 5, where the feedback comprises at least one of, a touch 
input, a typed input, a mouse input, a voice input and a facial expression input concerning 
the viewed scene. 
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19. The system of claim 1 8, where the feedback concerns a current scene and where 
the feedback comprises at least one of, a command to skip ahead in the playlist, a 
command to skip back in the playlist, a command to generate a new playlist, a command 
to find scenes similar to the current scene and a command to play a longer scene related 
to the current scene. 

20. The system of claim 1 5, where the playlist updater adds and/or removes a scene 
from the playlist based on at least one of, a usage data, a feedback command and a time 
stamp. 

21. A method for facilitating non-linear viewing of media, comprising: 
selecting one or more stored scenes to view; 

analyzing one or more first metadata associated with the one or more selected 
scenes; and 

generating a playlist of stored scenes by evaluating one or more relationships 
between the one or more first metadata and one or more second metadata. 

22. The method of claim 21 where selecting one or more stored scenes to view 
comprises at least one of, selecting a scene based on the presence of a face in the scene, 
selecting a scene based on the absence of a face in the scene, selecting a scene based on 
the presence of an item in the scene, selecting a scene based on the absence of an item in 
the scene, selecting a scene based on a voice in the scene, selecting a scene based on the 
absence of a voice in the scene, selecting a scene based on a mood of the scene and 
selecting a scene based on the theme of the scene. 

23. The method of claim 22 where analyzing the one or more first metadata 
comprises computing one or more similarity scores for one or more pieces of metadata 
that hold information concerning at least one of a present face, a present item, a present 
voice, a mood and a theme. 
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24. The method of claim 23 where generating the playlist comprises placing one or 
more scene identifiers in the playlist. 

25. A method for non-linear viewing of media, comprising: 

displaying a scene stored in a playlist storing at least one scene and metadata 
associated with the scene; 

receiving a user feedback related to the displayed scene; 
reacting to the user feedback; and 

if the user feedback commands an update to the playlist, selectively updating the 
playlist based, at least in part, on the user feedback. 

26. The method of claim 25, where receiving the user feedback comprises receiving 
at least one of, a touch input, a typed input, a mouse input, a voice input and a facial 
expression input. 

27. The method of claim 25, where reacting to the user feedback comprises at least 
one of, moving forward in the playlist, moving backward in the playlist, and displaying a 
media item related to the scene. 

28. The method of claim 25, where updating the playlist comprises at least one of, 
adding a scene to the playlist and removing a scene from the playlist. 

29. A method for facilitating non-linear viewing of media, comprising: 
receiving feedback related to a viewed media scene; 

if the feedback requests finding related scenes, finding related scenes by 
analyzing a metadata associated with the viewed scene; 

if the feedback requests generating a new playlist of media scenes, calling a 
method for generating a new playlist; and 

if the feedback requests navigating within a playlist, selectively skipping forward 
and/or backward in the playlist according to the feedback. 
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30. A system for annotating video media, comprising: 

a media database comprising a playlist and one or more video segments, where 
the video segments are associated with an annotating metadata; and 

an annotating tool that facilitates creating the annotating metadata and associating 
the annotating metadata with the video segments. 

3 1 . The system of claim 30, where the annotating metadata comprises at least one of, 
a date identifier, a time identifier, a videographer identifier, a face identifier, an item 
identifier, a voice identifier, a mood identifier and a theme identifier. 

32. The system of claim 30, where the annotating tool generates the annotating 
metadata in response to a user input. 

33. The system of claim 30, where the annotating tool automatically generates the 
annotating metadata based, at least in part, on a face recognition, an item recognition 
data, a voice recognition data, a mood data and a theme data. 

34. The system of claim 33, where the annotating tool is adapted based on a machine 
learning technique based, at least in part, on a user input concerning the annotating 
metadata generated by the annotating tool 

35. A system for delivering media content, the system comprising: 

a media data store comprising one or more metadata annotated, displayable items; 
a presenter that presents a selected first displayable item from the media data 
store; and 

a selector that selects a second displayable item from the media data store based, 
at least in part, on a relationship between a first metadata associated with the first 
displayed item and a second metadata associated with the second displayable item. 

36. The system of claim 35, where the media data store is at least one of a database, a 
data cube, a list, an array, a tree and a file. 
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37. The system of claim 35, where the presenter is at least one of an intelligent 
display and a non-intelligent display. 

38. The system of claim 35, where the presenter is at least one of an active display 
and a passive display. 

39. The system of claim 35, where the selector selects a second displayable item 
based, at least in part, in response to a user response to the first displayable item. 

40. The system of claim 39, where the user response is at least one of, a spoken word, 
a keystroke, a mouse click, and a facial expression. 

41 . A media annotation and retrieval system, comprising: 

an annotator that annotates a set of media items with a metadata to facilitate 
retrieving media items related by the metadata; 

a playlist generator that generates a playlist of media items related by a first 
metadata retrieved in response to a first query; 

a presenter for presenting media items associated with the media item playlist; 

and 

a playlist updater that updates the playlist based on a second metadata retrieved in 
response to a second query, where the second metadata is related to the first metadata. 

42. A method for annotating and retrieving media items, comprising: 
annotating a set of media items with a metadata to facilitate metadata based 

retrieval of one or more of the set of media items; 

receiving a first query related to identifying a media item by a relationship 
between the media item and a metadata; 

retrieving a first metadata responsive to the first query; 

generating a playlist of related media items where the media items are related by 
one or more metadata items in the first metadata; 
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displaying one or more media items listed in the playlist; 
receiving a second query related to identifying a media item by a relationship 
between the media item and a metadata; 

retrieving a second metadata responsive to the second query; and 
updating the playlist based on the second metadata. 

43. A data structure that facilitates non-linear viewing of media items, the data 
structure comprising: 

a first field that holds a media item; and 

a second field that holds a metadata item related to the media item, where the 
metadata facilitates at least one of identifying the media item, locating the media item 
and locating a related media item. 

44. In a computer system having a graphical user interface that comprises a display 
and a selection device, a method of providing and selecting from a set of graphical user 
interface elements on the display, the method comprising: 

retrieving a set of graphical user interface elements, each of the interface elements 
representing an action associated with facilitating the non-linear display of media items; 
displaying the set of interface elements on the display; 

receiving an interface element selection signal indicative, of the selection device 
selecting a selected entry from the set of interface elements; and 

in response to the interface element selection signal, initiating processing to 
facilitate non-linear viewing of media based, at least in part, upon stored metadata. 

45. A data packet adapted to be transmitted between two or more computer 
components that facilitate annotating a media and facilitate the non-linear viewing of the 
media, the data packet comprising: 

a first field that stores a clip identifier that identifies a portion of a media; 
a second field that stores a metadata key that identifies an annotating metadata 
associated with the clip identified by the clip identifier; and 
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a third field that stores data associated with the clip identified by the clip 
identifier. 

46. A memory for storing data for access by a computer component, the memory 
comprising: 

a data structure stored in the memory, the data structure holding: 

a first field that stores a clip identifier that identifies a portion of a media; 
a second field that stores a metadata key that identifies an annotating 

metadata associated with the clip identified by the clip identifier; and 

a third field that stores data associated with the clip identified by the clip 

identifier. 

47. A data packet adapted to be transmitted between two or more computer 
components that facilitate the non-linear viewing of a media, the data packet comprising: 

a first field that stores a clip identifier that identifies a portion of a media; 

a second field that stores a requested user action concerning the portion identified 
by the clip identifier; and 

a third field that stores metadata associated with the portion identified by the clip 
identifier, where the metadata is employed to adapt one or more clips according to the 
requested user action. 

48. A set of application program interfaces embodied on a computer-readable 
medium for execution on a computer component in conjunction with a system for 
annotating and viewing media, comprising: 

a first interface that receives media information; 

a second interface that receives annotation information associated with the media; 

and 

a third interface that receives user interface information associated with the order 
in which media will be displayed. 

49. A system for user-directed viewing of video, comprising: 
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means for partitioning a first video into one or more second videos; 
means for displaying the one or more second videos; and 

means for annotating the one or more second videos to facilitate retrieving at least 
one of the first video and the one or more of the second videos in response to a user input. 

50. A computer readable medium storing computer executable components of a 
system for media annotation and retrieval, comprising: 

an annotating component that annotates a set of media items with a metadata to 
facilitate retrieving media items related by the metadata; 

a playlist generating component that generates a playlist of media items related by 
a first metadata retrieved in response to a first query; 

a presenting component for presenting media items associated with the media 
item playlist; and 

a playlist updating component that updates the playlist based on a second 
metadata retrieved in response to a second query, where the second metadata is related to 
the first metadata. 

51. A computer readable medium storing computer executable instructions operable 
to perform a method for annotating and retrieving media items, comprising: 

annotating a set of media items with a metadata to facilitate metadata based 
retrieval of one or more of the set of media items; 

receiving a first query related to identifying a media item by a relationship 
between the media item and a metadata; 

retrieving a first metadata responsive to the first query; 

generating a playlist of related media items where the media items are related by 
one or more metadata items in the first metadata; 

displaying one or more media items listed in the playlist; 

receiving a second query related to identifying a media item by a relationship 
between the media item and a metadata; 

retrieving a second metadata responsive to the second query; and 
updating the playlist based on the second metadata. 
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